Analysing the Applicability of ChatGPT, Bard, and Bing to Generate Reasoning-Based Multiple-Choice Questions in Medical Physiology

Background Artificial intelligence (AI) is evolving in the medical education system. ChatGPT, Google Bard, and Microsoft Bing are AI-based models that can solve problems education. However, applicability of AI to create reasoning-based multiple-choice questions (MCQs) field physiology yet be explored. Objective We aimed assess compare generating MCQs for MBBS (Bachelor Medicine, Bachelor Surgery) undergraduate students on subject physiology. Methods The National Medical Commission India has developed an 11-module curriculum with various competencies. Two physiologists independently chose a competency from each module. third physiologist prompted all three AIs generate five chosen competency. two who provided competencies rated generated by scale 0-3 validity, difficulty, reasoning ability required answer them. analyzed average scores using Kruskal-Wallis test distribution across total module-wise responses, followed post-hoc pairwise comparisons. used Cohen's Kappa (Κ) agreement between raters. expressed data as median interquartile range. determined their statistical significance p-value <0.05. Results ChatGPT Bard 110 only 100 it failed them validity was 3 (3-3) (1.5-3) Bing, showing significant difference (p<0.001) among models. difficulty 1 (0-1) (1-2) (p=0.006). no (p=0.235). K ≥ 0.8 parameters Conclusion still needs evolve showed certain limitations. significantly least valid MCQs, while difficult MCQs.